Photo3D User's Manual (Win95/NT version)

Contents

Photo3D window
1. File
2. Calibration
3. Positionning
4. Modeling
5. Texture mapping
6. Preview
7. Coordinate
8. Tool

    Appendices

A. For better precision of calibration
B. Camera angle and precision
 

Photo3D window

The following window appears, when you invoke the Photo3D system.

figure I.1

Main Drawing Area is used to display photographs, 3D models and various kinds of objects. Above the Main Drawing Area, there are several interactive tools whose names are shown in the figure I.2.


figure I.2
 

Menu Tabs consists of seven menu tabs. Each tab corresponds to an operation mode. When you start Photo3D, the File tab is selected as default and its children menu buttons are shown.
 

 

Tool Bar consists of three menu buttons. They are used in common in multiple operation modes.
 

 

Prefix Window is used to modify behaviors of various kinds of operations by entering a numerical value in it. For example, by setting a value 0.01 in it the mouse cursor gains as 1/100 distance compared to the default setting value 1.0, thus enables you to operate precisely.
 

 

Thumbnail Window shows the list of photographs you are working with. Bright blue rectangle indicates that the enclosed photo is selected for the current work. The original counterpart is displayed in the main drawing area.
 

To change the current photo, click on the thumbnail you want to edit currently.
 

In Message Window prompt messages and error messages are displayed. Error messages are shown in red, accompanied with a sound of warning.

Example:

   or

 
 
Submode

Some of seven modes have its submodes. Menu buttons corresponding to submodes are grouped together and enclosed in a bevel to indicate they are submode group. Only one of them can be selected at a time.
 


 
 This is an example in the case of the Coordinate tab. These three buttons corresponds to
1. origin submode
2. axis submode
3. scale submode
 

 1. File

Project and related files

A project is a set of data used to create model by Photo3D. It consists of photo image data, calibrated data, 3D model and texture mapping data. Calibrated data include:
      1. relation between photograph and real world specified by calibration step
      2. relation among photographs specified by positionning step

A project is saved as a project file and related graphic image files, which are copies of source photo data and texture files. A project file has file name extension ".p3d". Related graphic files have extensions such as ".jpg" or ".bmp" depending on the format type. They are saved in the same directory as the project file and have automatically generated names which are related to the project file name.

 

Menu buttons in File tab

This menu adds a photo into the project. Clicking on the "Add Photo" button invokes the file selection dialog box to specify the file name of image data file. In the current version, files of BMP and JPEG format are supported for photo data. The format type of a file is recognized by its file name extension: ".bmp" or ".jpg".
 
The loaded photo is displayed in the main drawing area, and its thumbnail is added in the "Thumbnail Window". Thumbnail of last added photo is enclosed in bright blue rectangle indicating that it is the currently selected photo. To change the current photo, click on the thumbnail.
 

This button removes the current photo from the project. Needless to say, the original image data is kept unchanged as a file.
 

This button clears all the data in the project.
 

This button opens an existing project which has been saved by "Save Project" command. Clicking on this Open Project button invokes the file selection dialog box to select the project file to be loaded.
 

This button invokes the file selection dialog to specify the directory and file name to be saved. File name extension ".p3d" for project file is automatically added.
 
Copies of photo data and textures, if any, are also saved in the directory same as the project file. They have names related to the project file name.

For example, if you have added a photo with file name "photo1.bmp" and save the project as "proj1", then the project file is saved as "proj1.p3d" and copy of the source photo data as "proj1_photo1.bmp". If you have defined texture mapping, the texture files are saved as "proj1_001.bmp", "proj1_002.bmp", ...
 
 

Click on the button, then a dialog box appears to select the 3D format type to be exported. Current version supports VRML 1.0 and VRML 2.0. Click on OK button after the selection, then a file selection dialog appears to specify the file name of output. The extension ".wrl" for VRML file is automatically added.

If you have defined textures, they are output as JPEG files in the same directory as the VRML file with related file names. For example, if you export the project with name "vr1", then the texture files get the names "vr1_001.jpg", "vr1_002.jpg", ..., while VRML file is named "vr1.wrl".
 

In "1.5 Save Project", copies of the photo data are converted into a single format type, regardless of the type of each source data. The first photo determines the type of the rest, even if mixed type of source are used.

You can change this default type to another one, using "Options" dialog which is invoked by the Options button.

 

This button quits the program.
 
 

2. Calibration

The purpose of this step is to determine the orientation of the camera relative to the supposed coordinate system in the real world. We use graphical user interface called "frame lines" for the task.

We assume there are some hints in the photo which suggest the orientation of the supposed coordinate system in the real world. At least two lines are needed in the photo as hints for each of the three coordinate axes. In the real world the pair of lines must be parallel to the corresponding axis. The longer, the better. The more distant each other, the better.

 

 
Basic operations in Calibration Mode

Click on the Calib tab, then three pairs of line segments (six line segments) are shown against the background photo. We shall call the line segments frame lines, because they determine the framework of the 3D space. Each pair is colored in Red, Green and Blue respectively. Each pair corresponds to one of the three mutually orthogonal directions x,y and z.

 

figure 2.1	figure 2.2

 

Each line segment (frame line) has small circles at the ends and a short arrow near its midpoint. By picking one of the small circles and dragging it without releasing the mouse button, you can move an end point of a frame line. By dragging the short arrow, you can move the frame line parallel.
 

figure 2.3

For each direction of x, y and z, choose two lines in the image that are parallel to the direction. To be parallel, here, doesn't mean parallel on the screen, but parallel in the 3 dimensional world. Move frame lines to fit the chosen lines in the image.

Color usage for vertical and horizontal directions

Vertical direction must be specified by the blue pair of frame lines, while two horizontal directions are specified by either the red or green pair. Red and green are exchangeable.
 
 

Menu buttons in Calib tab



 There are seven menu buttons in the Calib tab.

 

Reset the frame lines to their initial position.
 
 

Sometimes the default color of a frame line is similar to that of the background image, and it is hard to distinguish. This menu button changes the color of frame lines so that they can easily be distinguished. There are three modes for changing the frame line color.
1. dark      for bright background
2. bright    for dark background
3. auto      automatically selected color
Clicking on the button causes cyclic change among the three modes.

The "auto" mode takes effect only when a frame line is dragged, and affects only on the pair which the dragged frame line belongs to.
 
 

There are four submodes in the Calibration mode:
1. Move Frame Line
2. Copy Frame Line
3. Delete Frame Line
4. Examine
The following grouped buttons correspond to the four submodes. Only one of them can be selected at a time.
 

Most operations can be done in this submode. You can move frame lines by dragging the small circles at the end of a frame line, or by dragging the short arrow near the midpoint of a frame line.
 
 

This button is used to increase the number of frame lines for a certain direction. For details, see Appendix A.

 

 
 This button is used to decrease the number of frame lines for a certain direction. For details, see Appendix A.

   2.6 Examine Mode
 

This button is used to examine whether the result of calibration also fits well to the lines not used in calibration. For details, see Appendix A.
 
 

 

This button invokes "Vanishing Point Window" which is used in case of using three or more frame lines to inspect whether they meet well at a point or not. For details, see Appendix A.
 
 

Tips for quick and precise fitting

   Magnet Mode

For quick fitting, turn magnet mode on by the magnet button. In the mode, Photo3D searches an edge in the background image near the current position of the frame line, and automatically modifies the position of the frame line to fit the detected edge. The modification occurs when the mouse button is released. This mode can be turned off by clicking on the button once more.

Mouse Sensitivity

For precise positionning of frame lines, decrease the mouse sensitivity by entering a small value in the prefix window. For example, by setting value 0.01 the mouse cursor gains as 1/100 distance compared to the default setting where the prefix value is set to be 1.

   Magnifier

To see a magnified image, activate the magnifier window by pushing the magnifier button.  The area to be magnified is indicated by a red rectangle in the main drawing area. The area can be moved by picking and dragging the boundary of the red rectangle. This window can be turned off by clicking on the magnifier button once more.

   Stretcher

In some cases it is difficult for the magnifier to see a magnified image of a long line. In such a case, please use stretcher instead of magnifier.

Stretcher is an another type of magnifier. The rectangle to be magnified can be placed obliquely in the main window, and this region is mapped into a square in the auxiliary window. Thus a thin rectangle along a long line is shrinked along the line and stretched in the perpendicular direction.

It is the perpendicular direction that matters when you are trying to fit a frame line to an edge in the background image. Thus this tool enables you to check the fitness of a long frame line.

This tool is also used to correct the lens distortion. For more detail, see 8.3 Stretcher.

3. Positionning

When is this step needed?

If you create a 3D model from only one photo, you don't need this step. You can go to modeling mode immediately after finishing calibration. However, if you use two or more photos, this step is needed in order to keep consistency among multiple camera angles.

Even in the case of multiple photos, the first photo can be processed without this step. Modeling can be done after calibration. Create a few polygons using the first photo. Then for each additional photo, do the positionning before modeling.
 

Specifying positional information

In calibration step, we have determined merely the directions of the three axes of the coordinate system. We have specified nothing about the origin nor coordinate value of any points. Thus, we have to give clues to the program, so that it can display the 3D model in a correct way overlapped against each photo as background.

For example, Figure 3.1 is an instance of modeling process. Two polygons have been created using a photo of the upper half of a building.


Figure 3.1

Let's move to the second photo to create the lower half of the building. If we go to the modeling mode immediately after calibrating the second photo, a scene like Figure 3.2 is displayed. The two polygons of the upper half is shifted aginst the background image as opposed to the correct position as shown in Figure 3.3. Correcting this shift is one of the purposes of this step.
 

Figure 3.2

Figure 3.3

 

 

Orientation of the model

Suppose we have corrected the shift and added two polygons as the lower half of the building as shown in Figure 3.4.

 
Figure 3.4

If we try to work with the third photo of the building taken from a different angle, a scene like Figure 3.5 will be shown.

It is not only shifted but also rotated 90 degree compared to the right position as shown in Figure 3.6. It is because we have not specified in the calibration step, which is x, which is z. Only the vertical direction (here, we call it y-direction) has been specified using the blue pair of frame lines according to the convention.

Putting the orientations in order is another purpose of the positionning step.
 
 

Menu buttons in Positionning Tab

   3.1 Rotation around vertical axis

Click on the rotation button. Then the 3D model rotates 90 degree around the vertical axis. Go on clicking until the orientation of the model matches the background image.

   3.2 Parallel to screen move

After adjusting the orientation of the model, go to screen-parallel-move submode by this button, where dragging the mouse causes the model to move parallel to the screen. Choose a characteristic point (for example a corner of a wall) of the model in mind, then move the model so that the characteristic point on the model matches to the corresponding point on the image.
 

   3.3 Depth move

Push the select-target button  , and pick the characteristic point of the model which you chose above, then a small mark is attached to the point to show the point is selected. Next push the depth-move button. Then you go into depth-move mode, where dragging mouse causes the model to move in the depth direction, keeping the selected target point stable. Pulling the mouse toward you brings the model near to you. Adjust the size of the model to fit the background image.

4. Modeling

The task of this step is creating a 3D model with the aid of background image. Needless to say, we must specify 3 dimensional information, rather than merely tracing on the image two dimensionally. A user interface element named "#-cursor" will help you to perform this task using the calibrated data.

The "#-cursor" is a rectangle supposed to be placed in the 3D world space and displayed on the screen overlapped against the background image. We can move each of its four sides independently within a plane which supports the cursor. It is also possible to rotate the "#-cursor" around one of its sides as a rotation axis. Thus these functions enables us to locate the cursor anywhere in the 3D space.

   4.1 Initializing "#-cursor"

Clicking on the Initialize #-cursor button  brings the cursor near the center of the screen. It is oriented so that two of its sides are parallel to x-axis, and other two parallel to y-axis (vertical axis), thus the supporting plane parallel to xy-plane.

   4.2 Edit Cursor

Confirm that you are in the "Edit Cursor" mode as shown below.

If not, press the button to get into the mode. Then, pick one of the four sides of the "#-cursor" and drag it without releasing the mouse button. The picked side moves parallelly in the 3D space.

   4.3 Rotate Cursor

You can rotate the "#-cursor" around one of the four sides as rotation axis. Push the Rotate #-cursor button, then click on one of the four sides of the "#-cursor". It rotates around the picked edge. The angle of rotation is 90 degree by default. Instead you can rotate any amount of angle by setting a value in degree in the prefix window. Negative value means the opposite direction of rotation.

Adjusting "#-cursor" to an existing element

Edit cursor mode allows you to move edges of "#-cursor" in free hand precision. However sometimes it is necessary to move edges to an exact position relative to existing model elements. The following two functions help you to do such operations.
 

   4.4 Move to existing point

This operation  consists of three steps:

Push the Move to point button.

Pick one of the sides of the "#-cursor" which you wish to move.

Pick an existing point where you wish to carry the side to.

If the picked point lies on the same plane as the #-cursor, the side moves parallelly so that it (or its extension) passes through the picked point. If the point is not on the plane of the #cursor, the edge is moved so that it passes through the foot of perpendicular line from the picked point to the plane.

   4.5 Jump to existing line

This operation also consists of three steps:

Push the Jump to line button.

Pick one of the sides of the "#-cursor" which you wish to move.

Pick an existing line to where you wish to carry the side.

The picked line must be parallel to the picked edge of the cursor. ("Existing line" is a side of existing rectangle or polygon.)
Unlike the "Move to existing point" above, this operation carries the whole cursor as a rigid body. The target line must be parallel to the picked side of the "#-cursor".
 

Creating elements

Menu buttons 4.1- 4.5 above are all related to positionning of the "#-cursor". Buttons which actually creates elements are explained below.

   4.6 Create rectangle

Push the Create Rectangle button. Then a rectangle is created where the "#-cursor" is placed.

   4.7 Create point

Push the Create Point button. And pick one of the four corners of the "#-cursor". Then a point is created there.

   4.8 Create Polygon

Push the Create Polygon button. Then pick existing points successively to form the vertices of the polygon to create. Picking the first point again causes the polygon to close. These points must lie on a plane in 3D space.

   4.9 Delete Polygon

Push the Delete Polygon button. Then pick the polygon to be deleted.

5. Texture mapping

 

5.1 Defining Texture

Pick one of the polygons of the model in this mode, then it gets marked with translucent red indicating that it is given a texture. The texture data is sampled from a portion of the current background image, and converted as it is viewed from the normal direction of the polygon. Textured polygons defined by another photo are marked in translucent blue.

   5.2 Texture Resolution

You can control the texture resolution for each polygon in the model respectively. Higher resolution causes the polygon to be displayed in finer texture mapping, at the expence of system resources; larger size of texture file, larger amount of memory and longer time to display.

There are two ways for specifying the texture resolution:

• by texture size
• by physical resolution
 
In the following explanation, we use the term "texel" for a pixel in a texture image.
 
By Texture Size

A texture is a rectangluar shaped image data which covers the polygon in question. Texture size is  the width and height of the rectangle measured in number of texels. Larger texture size enables finer texture mapping.

By Physical Resolution

Here, "physical" means the world coordinate system in which the 3D model is defined. Physical resolution is the number of texels contained in unit length of the coordinate system.

A texture defined by this way has the proper aspect ratio. For example, a square window placed on a wall is square also on the texture image data. Moreover, textures defined using the same resolution have the same scaling factor. Thus a window on a blurred texture can be replaced by a window of the same size on another clearer texture, if any, using an image editing software**. You can make a paper model of a building by printing out the correctly scaled textures, for example.

** For example if your system is in Windows standard configulation, "Paint" can be used as the tool for this purpose. It is invoked from Start-menu/Program/Accessories.
 

Indicators

There are several indicators to help determining the appropriate value of resolution:
 
Effective resolution
Effective resolution at a point on the model is the resolution value equivalent to the resolution of the current background photo on the screen. Effective resolution depends on the depth of the point in question. Nearer point has larger value of effective resolution. It is the maximum resolution to exploit the resolution of the background photo for the texture around the point.

For more precise description of effective resolution, suppose a line segment of unit length which is parallel to the screen. Effective resolution is the length of this line segment measured in number of pixels on the screen.

Effective resolutions at the nearest point and at the farthest point are shown on the dialog box as indicators. A resolution value which exceeds that of the nearest point only wastes system resources without improving the image quality.

Largest texture
The sizes measured in texels of the widest, the highest and of maximum area (width x height) are shown as indicators.

Texel Size
Texels can be thought as square tiles to be spread over to fill the surface of a polygon. Texel size is the dimension of a tile measured in world coordinate system. It is the reciprocal number of the physical resolution "texels per unit length".

 
2ⁿ round up

"2 ⁿ round up" option is intended to be used for OpenGL applications which require texture sizes to be power of 2. The calculated size for a texture is rounded up to the nearest larger integer of power of 2. If the calculated texture size for a polygon is 65 x 150, for example, it is rounded up to 128 x 256. The whole rectangle area is used as texture image in order to get the maximum resolution. Thus the aspect ratio is no longer kept.

 
 
Figure 5.1 Texture Size Dialog

Scale and resolution

If the scaling (see 7.3 Scale) is altered after the texture has been defined, it should not be expected that the specified resolution (number of texels per unit length) is kept any more. The resolution is not an attribute of the texture, but is only used to determine the texture size together with the current scaling when the polyogn is defined a texture. Once the texture is defined, only the texture size is kept as the attribute of the texture of the polygon, not the resolution. Thus the apparent resolution (degree of detail) is kept unchanged.

    5.3 Texture File Format

This menu affects the texture format both by "Save Project" and by "Save Texture" described below. This choice has no effect on the texture format of exporting VRML --- it is always JPEG.
 
Push the Format button in the Texture tab. A dialog box appears to specify the graphic format of textures to be saved. In the dialog, letters are colored in green to avoid the confusion with the dialog invoked by the Option button in the File tab.
 
Unlike the "Size" menu, the selection is not modal, but it specifies the behavior of saving action for all the textures (not the format for each one of the polygons).
 
 

     5.4 Saving Textures

Usually you don't need to use this menu to save the textures. Instead, they are automatically saved together with the project file. This menu is intended to facilitate saving a set of textures in specified name.

6. Preview

The created 3D model can be examined by various viewing operations in this "Preview" mode.

   6.1 Rotation

Go to the rotation mode by pushing the Rotation button.  Press the mouse button and drag the mouse without releasing the button. By the horizontal movement of the mouse cursor, the model rotates around the vertical axis. By the vertical movement of the mouse cursor, the angle of depression varies.

   6.2 Horizontal scroll

Go to the horizontal-scroll mode by pushing the Horizontal-scroll button. The model moves as the mouse moves.

   6.3 Updown

Go to the updown mode by pushing the Updown button.  By the vertical movement of the mouse cursor, the model moves up or down.

   6.4 Look around

Go to the look-around mode by pushing the Look-around button. By moving the mouse freely, you can look around with your eye position fixed.

   6.5 Zoom

Go into the zoom mode by pushing the Zoom button. By moving the mouse cursor up (or down), the model is displayed zoomed out (or in).

   6.6 Wire frame mode or texture mode

The model is displayed in the wire frame mode by default. Push the mode button. Then the button face changes showing that the mode is changed into texture mapping mode.

7. Coordinate

Photo3D's modeling generates a set of 3D coordinates.  The following is an excerpt from an exported file of VRML 2.0 output.
.....
point [
   -92.914   27.574   86.816, #0
   -92.914   27.574  225.996, #1
   -92.914  177.207  225.996, #2
   -92.914  177.207   86.816, #3
 ]
If you wish to control the coordinate system for output, specify it by the "Coordinate" mode before exporting file. The following is the same portion of the VRML file with coordinate specified.

 point [
     0.000    0.000    0.000, #0
     0.000    0.000    1.000, #1
     0.000    1.068    1.000, #2
     0.000    1.068    0.000, #3
]
 
Photo3D's coordinate mode enables you to specify the origin, direction of axes and scaling.

Push the Origin button, then click on a point in the model which you wish to be the origin.
 

Push the Axis button. There appears a cross colored in orange, indicating the  four horizontal directions as the candidates of the x-axis. Click on one of them, then it turns into the x-axis. Y-axis is fixed to be the vertical direction. Z-axis is calculated from x, and y-axis.
 

Scaling is done by specifying the distance between two chosen points.

Push the Scale button, then you are prompted by a message "Pick the first point" in the message window. Pick one of the points in the model. Then pick the next point according to the prompt message "Pick the second point". If you have successfully picked the two points,
a dialog window to specify the dimension appears.
 

8. Tool

   8.1 Magnet

This tool can be used in calibration mode and modeling mode, in order to facilitate the mouse operation to put objects such as frame line or "#-cursor" to target edges in the background image. To activate the magnet, push magnet button.

When you are carrying a frame line in Calibration mode, the magnet automatically finds an edge in the background image near the frame line and moves it to the detected edge. Similarly, when you are moving a side of the "#-cursor" in the modeling mode, it searches an edge and moves the side of the "#-cursor" to the position of detected edge in the image. But in modeling mode, as the side of the "#-cursor" must move parallel, the direction of the side is modified to keep parallel.

Be careful to inactivate the magnet when you don't need it, otherwise you will get an unexpected movement of the frame line or the "#-cursor". To inactivate the magnet, push the Magnet button again.

   8.2 Magnifier

Magnifier maps a small rectangular portion of the main window into the auxiliary window to get magnified image. To activate the magnifier, push the Magnifier button. The portion to be magnified is indicated by a red rectangle in the main window. To move the magnified region, pick an edge of the red rectangle and move the mouse without releasing the mouse button. Magnifier function is available in every mode except preview mode.

   8.3 Stretcher

In some cases it is difficult with the magnifier to see a magnified image of a long line. In such a case use Stretcher instead of Magnifier.

Stretcher is another type of the magnifier function. It magnifies unisotropically; It has two different magnifications in particular two directions which are mutually orthogonal. In a typical case, it magnifies in one direction, and minifies in another direction. With this tool a thin rectangle along a long line in the main window can be compactly mapped into a square of moderate size in the Stretcher Window without decreasing the magnification factor in the required direction.

To activate, push the Stretcher button. Then an auxiliary window with a tool bar on the top appears, while, in the main window, there appears a blue rectangle indicating the region to be mapped into the auxiliary window. A red center line with arrow heads is attached to the rectangle. By picking an arrow head at the end of the center line and dragging without releasing the mouse button, you can move the center line together with the blue rectangle. By picking and dragging the short arrow at the middle of the center line, you can move the center line parallelly. By picking and dragging a side of the rectangle which is parallel to the center line, you can alter the width of the rectangle.

There are two display modes in the auxiliary window. When  (the Distortion button) is pushed down, blue rectangle and red center line corresponding to that of the main window are displayed. On the other hand, when  (the Frame line button) is pushed down, frame lines and wire frame of the 3D model, if any, are displayed.

Stretcher and lens distortion correction

Stretcher has an exaggerating effect of lens distortion. Suppose there is an image with a slight lens distortion. Choose a line in the image, and place the stretcher's region indicator (blue rectangle) along the line.(Figure A) The corresponding image in the auxiliary window shows the region with exaggerated distortion.(Figure B) A line which is slightly curved in the main window is displayed with so much curvature that the distortion is easily visible.

The amount of lens distortion can be expressed by a parameter called distortion coefficient. Our task is to estimate the value of this distortion coefficient, and to correct the image according to the estimated value of distortion coefficient.

Estimation of distortion coefficient



 Enter an arbitrary value of distortion coefficient (such as 0.1 for example) in the text window, then the red center line gets distorted as shown in Figure C. Regulate the value so that the curved red line fits the target line in the image.

Figure A	Figure B	Figure C	Figure D

Correction of the image

After you get an appropriate estimated value of the coefficient in the text window, push  (the correction button). The background image is corrected so that the red curve gets straight. The corresponding image in the auxiliary window also gets corrected as shown in Figure D.

The corrected image is saved when you save the project, and automatically loaded with the model when you load the project next time. The file name for the corrected image is
 
"ppp_fff_cor.xxx"
 
where ppp is the project name, fff is the photo name and xxx is "jpg" or "bmp" according to the file format type. For example, if you add a photo "photo1.jpg" then correct the lens distortion and save the project as "proj1", the name of the saved photo is
 
"proj1_photo1_cor.jpg"
 
 and it is automatically loaded, when you open the project "proj1.p3d".

Appendices

A. For better precision of calibration

Examining mode

For each direction of x, y and z, we have chosen only two lines in the image. However, in general, there are many lines parallel to the direction. For example, walls of a building have many parallel lines made up of window frames. In examining mode, we can check whether the the direction determined by the two lines matches also with the remaining lines.

In order to examine one of the three directions, turn examining mode on, by pushing the examining button. Pick one of the frame line to indicate which direction you are going to examine, then drag the mouse without releasing the mouse button. This operation does not affect the picked frame line. Instead a new line segment will appear with one end at the mouse cursor, another end at the vanishing point --- cross point of chosen two lines. Move the cursor so that the new line segment overlaps one of the remaining lines which you wish to examine. This mode can be turned off by pushing the button once more.

Determining by choosing more than two lines

We used only two lines to determine the direction. In examining mode mentioned above, other lines parallel to the direction were used only for checking the result. By letting these lines participate to determine the direction, we can expect more stable estimation.

To accomplish the task we need the following steps:

Add another frame line

Enter into the copy mode by pushing  (the copy button).   Pick one of the frame line which belongs to the direction you wish to add another frame line. Then a new frame line will be created, which is invisible at first because it is created at exactly the same position as the picked one. The mode automatically turn from the copy mode into the usual mode (mode to move frame lines).

Move the added frame line

 

Move the added frame line by usual operation to one of the remaining line in the image which is parallel to the direction in question. Since the frame lines, including the added one, are parallel in 3D space,  they should cross each other at a single point --- vanishing point. Or, in geometric terminology, they should be concurrent. However they are not exactly concurrent because of various kind of errors, most likely because of error of fitting. In the following we focus on the fitting error and try to eliminate, or decrease at least, this kind of error. (There are other kind of errors; For example, strain of image typically caused by lens distortion, and dimensional error of the object in the world itself.)

 

Check the concurrence

 

Activate the concurrence window by pushing  (the concurrence button). There appears an auxiliary window, which shows the situation near the (estimated) vanishing point. A small yellow square represents the estimated vanishing point. Horizontal and vertical lines in gray show the size of pixels. In this window, we can examine how far the frame lines lie from concurrent.

 

 
 
Delete frame line

To delete a frame line, push  (the Delete button). then pick a frame line to be deleted. If the number of frame lines in a direction gets 2, you cannot delete any more.
 

 B. Camera angle and precision

In most photographs, the camera angle is hold horizontal and as a result vertical lines appears almost parallel on the photo. It is natural as a photograph, but it is not good for Photo3D's calibration algorithm. Parallel (or almost parallel) pair of frame lines makes the cross point very far, and injures the precision of calculation. A slight depression or elevation of the camera angle will improve the situation. Suppose three mutually perpendicular edges meet at a point. The best angle is the angle in which they meet in 120 degree on the photo.

  ideal camera angle

This tendency of the algorithm shall be improved in the next version of Photo3D.